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Finding the Answers in Research: 

An Interview With Kiley Walsh 



I n her groundbreaking study, titled 
After theTest: H ow Schools A re Using Data 
to Close theAchievement Gap, researcher 
Kiley Walsh shows clear evidence of 
specific work being done to close 
achievement gaps. Walsh selected 32 
schools across six counties in the San 
Francisco BayArea and compared those 
making progressin closing achievement 
gaps with those not making progress. 
Her findings speak with clarity about 
issues of race, leadership, school focus, 
support for teachers, and the critical 
role of using data to improve achieve- 
ment for all students. 

MSAN recently spoke with Walsh about 
the research she has conducted and 
practical ways of applying that research 
in schools. 

MSAN: What made the schools you studied 
special? Why were they able to do things so 
many others haven't? What were some of the 
key elements that madethese schools succeed? 
What are the implications of this work for 
other schools and other grade levels? 

Walsh I think there were three impor- 
tant elements. First, high-quality data 
and quick access to results. Teachers 
need news they can use: quick oral flu- 
ency exams, diagnostic assessments 
given every six weeks with a fast turn- 
around reporting loop. This is ideally 
supported by the district. 

Second, support for changes in instruc- 
tion. These were schools— in some 
cases led by district reforms— that 



invested heavily in peer collaboration, 
coaching, and professional develop- 
ment so teachers could act on data. 

Third, leaders who aren't afraid to 
talk about race, have courageous 
conversations about achievement gaps, 
and have a sense of cultural proficiency. 
Frequently, the most vocal and active 
parents in a community represent the 
children who are the most empowered 
and who are doing well, comparatively. 
For a leader to champion the needs of 
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FROM THE executive director 

To meet the goals set by our districts, the 
Minority Student Achievement Network (MSAN) 
is attempting to discover through research what 
works to eliminate the achievement gaps. With 
the assistance of funding from the George Gund 
Foundation, MSAN has developed and is testing 
a validation procedure to identify those promising 
practices at work in our districts and to validate 
these practices as proven. Through the generous 
contribution of a grant from the Spencer 
Foundation, in partnership with Uri Triesman, 
Ph.D., and researchers at the Charles A. Dana 
Center of the University of Texas at Austin, we 
are developing strategies to increase algebra 
course taking and success. Funds from the W. K. 
Kellogg Foundation helped support the annual 
conference in June 2005, where we continued to 
disseminate what districts are learning. And 
funds from the Nellie Mae Education Foundation 







and a generous benefactor helped fund our 6th 
annual student conference, where we were able 
to listen to students and hear firsthand their 
suggestions for closing the gaps. 

In addition to learning from what is successful 
within MSAN districts, we also are learning from 
research conducted in school districts nation- 
wide. In 2002-03, the North Central Regional 
Educational Laboratory provided funding for the 
BayArea School Reform Collaborative (BASRC) to 
conduct a study of schools in the San Francisco 
BayArea that are closing the achievement gaps 
and those that are not. The study, conducted by 
Kiley Waish, found several distinguishing fea- 
tures of gap-dosing schools: the frequent use of 
data for classroom decision making, teachers 
who visit the classrooms of peers to observe 
practice, school and district leadership that 
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(continued from pagel) 

the bottom part of the gaps— they're 
doing so at the risk of alienating a 
vocal political support base. It takes 
courage and moral vision to do this. 
Some leaders in schools I studied, 
while frustrated by the difficulty of 
meeting all adequate yearly progress 
requirements, candidly revealed that 
NCLB [No Child Left Behind] gave 
them momentum to do what they'd 
always wanted to do. Now they could 
push for equity and use "the Feds" as 
a reason with parents who pushed back. 

MSAN: In your study After the Test, you 
suggest that data can beused as a catalyst 
for effective work leading to change and 
improvement in school culture. What does 
this process look like, and what are the 
steps to making it happen? 

Walsh: Change happens when data 
are useful to teachers. Data from an 
annual assessment can meet the 
needs of the public or policymakers- 
and these are important accountability 
needs- but big-picture data, which 
teachers don't see until the following 
fall after students have moved on to 
other classrooms, don't inform daily 
instruction. In contrast, benchmark 
or diagnostic assessments linked to 
curriculum and given throughout the 



school year can provide useful infor- 
mation about student skill gaps that 
inform daily practice and give teach- 
ers an effective tool to work with stu- 
dents who are performing below 
grade level. 

In a lot of schools, looking at data is 
something imposed from external 
forces and used in a punitive way. 
Teachers are beaten over the head 
with it but not given any resources or 
new strategies to change what they're 
doing. The result is frustration and 
burnout. I n schools where data are a 
catalyst for change, teachers are given 
time to look at data and support to 
learn new strategies through coach- 
ing, modeling, and demonstration 
lessons so they can meet identified 
needs. This makes the data a catalyst 
for change: Teachers are given the 
means to be efficacious. 

One school I studied had each 
teacher conduct a "classroom-level 
cycle of inquiry/' selecting one or two 
students from within the lowest per- 
forming subgroup for the whole 
school. In this case, it was Hispanic/ 
Latino boys. Teachers talked about 
initially resisting thistask; intentionally 
reflecting just on a couple of students 
seemed to go against the charge to 
teach all kids. Many then realized, 
however, that they hadn't been meet- 



ing these kids’ needs. They felt that 
data-based reflection made them bet- 
ter teachers— not just for the focal 
students but for all. 

MSAN: In the schools that you studied, 
what was the response of teachers to an 
emphasis on using data? Did schools 
engage in specific activities and strategies 
to ensure all teachers were on the same 
pagein terms of using data? 

Walsh: Most gap-closing schools I 
looked at had initial resistance, fol- 
lowed by acceptance, and then an 
embrace of data. I don't want to over- 
state the "embracing"— in general, 
teachers still felt they had to do too 
much with too little— but they came 
to see the tremendous value of data in 
helping them become better teachers 
and helping all students learn. 

To get teachers "on the same page" in 
terms of data use, a lot of the gap- 
closing schools invested in teacher 
collaboration, setting aside time during 
the weeklyschool schedule for teachers 
to meet in grade- or content-alike 
groups to discuss data-related strate- 
gies. One common activity was cali- 
bration around grading, or agreeing 
upon and codifying what "A" work 
looks like, what "B" work looks like, 
etc. That way, not only do the teachers 
get into alignment, but outcomes also 
can be explicitly communicated to 
the students. 

MSAN: You say in your study that "race 
matters" and that structured conversa- 
tions about race need to occur as part of 
the work to close achi a/ement gaps. What 
do thse conversations look like? Who's 
involved, and how do th^ work? How 
do these conversations link to improved 
student achia/ement? 

Walsh That recommendation stems 
directly from both the survey and 
case-study findings: that race is a cen- 
tral aspect of the achievement gaps, 
with African-American and Hispanic/ 
Latino children consistently perform- 
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ing at lower levels than their white or 
Asian-American peers. Over 90 per- 
cent of teachers in gap-closing 
schools reported that they had oppor- 
tunities for structured conversations 
about race/ ethnicity; the majority in 
non-gap-closing schools did not, with 
over 40 percent strongly disagreeing 
that they had opportunities. 



i think there are two key points here. 
The first is the word "structured." 
These are not free-form conversa- 
tions, gathering staff together and just 
asking them to talk about race. 
Rather, these are facilitated conversa- 
tions around specific, data-based 
issues in their school. In one case- 
study school, the principal did focus 
groups with parents, students, staff, 
and community members and found 
widespread unhappiness with the dis- 
cipline system. The suspension rate 
was high. When the principal disag- 
gregated the data, she found that the 
majority of kids getting suspended 
were African American and three of 
every five African-American children 
had a suspension record. With struc- 
tured, data-based conversation on this 
topic, coupled with coaching from a 
veteran teacher around classroom- 
based discipline and the firing of two 
school security guards, not only was 
the overall school suspension rate 
reduced, the African-American sus- 
pension rate was radically reduced 
to under a third of the former rate, or 
18 percent. 

The second aspect to this, which I 
touch on above, is that gap-closing 
schools grounded their conversations 
in data. It wasn't about finger point- 



ing and generalizations; it was about 
teachers having their classroom-level 
data in front of them and being given 
structured time to talk about it, reflect 
on why it was happening, and then 
being given professional develop- 
ment, coaching, and collaboration 
time to change instruction and 
patterns of teaching and learning. 



MSAN: The recommendations of your 
study include that schools need to deter- 
minetheir focus. H ow do schools arriveat 
the item or items they need to focus on? The 
schools you looked at focused on literacy. 
Does literacy appear to be the first step in 
terms of focus? If so, how do you see this 
playing out with teachers who teach the 
subjects of math, science and social studies? 

Walsh Many schools are trying to do 
too many things all at once. Schools 
that were successful in closing gaps 
narrowed the list to the areas of 
greatest need or areas of greatest 
impact. For most, thismeantan inten- 
sive focus on literacy. Many schools 
made room for at least two hours of 
literacy instruction every day. The 
data in my study were not conclusive 
about what type of literacy approach 
works best. But it was clear that link- 
ing resources and support for 
improved instruction directlyto areas 
of greatest need— which is frequently 
literacy— can have a tremendous 
impact on student learning 

Teachers whose background is not in 
reading/ language arts frequently feel 
resistant to this focus. In my research, 
I've seen one of three things happen: 
They leave that school and find 
one where they're not being asked to 
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communicates constantly the importance of 
closing the gaps, and a thorough focus on a 
few initiatives. 

The report of this study, titled After the Test, 
demonstrates how schools and school 
systems can productively learn from their 
common experiences and strengthen their 
capacity to grow and learn as organizations. 
It also informs public policymakers, as well 
as educators and their supporters, about 
practice- informed applications of policy and 
innovations to improve educational out- 
comes. Such information is important 
because schools across the nation are expe- 
riencing increases in their populations of 
African-American and Latino students. 
Districts that had little or no ethnic diversity 
a decade ago are experiencing changes in 
their student populations. No Child Left 
Behind requirements to disaggregate data 
and measure results based on effectiveness 
to improve achievement for all students are 
pressuring districts to look for ways to effec- 
tively close the gaps in achievement for 
students of color. 

This issue of Network News features an inter- 
view with After the Test researcher Kiley 
Walsh. Her study is an outstanding example 
of contemporary educational research that 
has practical implication for schools. The 
findings have been widely quoted and 
shared. MSAN has doselyfollowed the BASRC 
work, and the Research Practitioner Council 
made the report the focal topic of its February 
2005 meeting. 

Also included in this issue is an overview of 
the Learning Point Associates report 
Perspectives on the Caps: Fostering the 
Academic Success of Minority and Low- 
Income Students. This report, also funded by 
NCREL, offers summaries of four research 
studies about strategies to eliminate the 
gaps in school achievement. After you read 
Ray Legler’s overview, I hope you will want to 
read the full report. ■ 

- Rossi Ray-Taylor 



"Gap-closing schools invested in teacher 
collaboration, setting aside time during the weekly 
school schedule for teachers to meeL.to discuss 
data- related strategies." - Kiley Walsh 
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income at the University of Maryland- 
Baltimore County. The goal of this 
study was to understand what worked 
in this particularly successful higher 
education program that may be rele- 
vant to K- 12 schools. 

The findings indicated that several key 
components of this program con- 
tribute to its effectiveness. First, the 
program emphasizes the academic 
and social integration of students into 
the student population, through the 
support of program staff. Second, the 
program emphasizes knowledge and 
skill development through participa- 
tion in faculty research and study 
groups. Support and motivation are 
provided through peer and staff men- 
toring, Finally, intensive monitoring 
and advising contribute to student suc- 
cess. The program has been extremely 
successful in producing minority grad- 
uates in mathematics and science. 

A Study on Closing Achievement Gaps 
in Rural Communities 
This study focused on efforts to reduce 
or eliminate achievement gaps in rural 
Michigan. The goal was to learn about 
effective strategies that school districts 
in rural areas can use to improve the 
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Recent Research 
on Closing the 
Achievement 
Gaps 



ByRayLegler, Ph.D. 



Research studies examine the issue from variety of 
perspectives and identify promising practices for 
closing the gaps. 



A Study on Improving Students' 
Literacy Skills 

This study investigated the impact of 
a program designed to improve the 
literacy skills of students of color and 
lowincomein an MSAN high school in 
suburban Chicago. The goal was to 
determine the effectiveness of the 
program and, if found to be effective, 
provide a model for literacy interven- 
tions in other settings. This study 
employed a quasi-experimental longi- 
tudinal design, with measuresof student 
reading ability at several time points. 



as well as instructional strategies and 
materials designed to be relevant to 
students and increase their interest in 
the material. Results of the study were 
mixed, but some were promising and 
suggest that the program warrants 
replication and further research. 

A Study of Student Supports to 
Improve Academic and Personal 
Success 

This study looked at the Meyerhoff 
Scholars Program, a program that 
supports students of color and low 



n 2003, North Central Regional 
Educational Laboratory (NCREL) 
funded four studies on strategies 
for closing the achievement gaps. Our 
goals were to examine the issue from a 
variety of perspectives and to identify 
promising practices for closing the 
gaps. The resulting publication, 
Perspectives on the Gaps: Fostering the 
Academic Success of Minority and Low- 
Income Students, presents summaries 
of the studies— including the study 
conducted by Kiley Walsh, which 
compared gap-closing schools to 
schools that were not closing the 
gaps in the San Francisco Bay Area. 
The other three studies discussed in 
Perspectives on the Gaps are as follows: 



The program involved identifying stu- 
dents who were performing poorly on 
tests of reading ability and providing 
them with an opportunity to partici- 
pate in a literacy course at the high 
school level. Students participated asa 
cohort and took other classes together 
in order to increase peer support. The 
program also emphasized teacher sup- 
port and encouragement for students, 
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Recent Research on Closing 
the Achievement Gaps 

(continued from page 4) 

academic achievement of low- 
income students and students of 
color. The study was exploratory 
and employed qualitative meth- 
ods to gain an understanding of 
current programs and policies in 
a group of rural schools and dis- 
tricts. It was funded largely 
because research in rural areas is 
limited, because the gaps are typi- 
cally discussed in terms of urban 
and suburban schools, and 
because a substantial portion of 
the NCREL region isrural. 

The study explored the issues of 
low achievement in rural districts, 
which largely reflect the low- 
income status of rural students 
(as opposed to the issues faced by 
students of color) . Results suggest 
that funding for rural schoolsand 
districtsisfar below what is needed 
to provide a quality education for 
rural students; often this is due to 
the low property-tax base in these 
areas. Results also suggest that 
administrators at the district level 
could do a better job of improv- 
ing education in their districts, 
and that professional develop- 
ment for these administrators 
may be helpful. ■ 

- RayLegler, Ph.D., is a senior program asso- 
ciate in NCREL's Center for Educational 
Decision Support Systems at Learning Point 
Associates. His work focuses on issues related 
to K- 12 achievement gaps. 

The full report, Perspectives on the Gaps: 
Fostering the Academic Success of Minority 
and Low-Income Students, edited by Ray 
Legler, is available online at www.ncrel.org/ 
gaplstudieslperspectives.pdf. 
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teach reading; they find a way to keep 
doing business as usual but make it 
appear as if changes are being made; 
or, in schools that invest in classroom- 
based support (such asliteracy coaches 
who can link instructional changes to 
individual content), I've seen science, 
social studies, and math teachers 
empowered, excited, and enlivened by 
new strategies to help their students 
comprehend subject-specific material. 

Again, it's about ownership: Once 
teachers feel they're really connecting 
with students more effectively and 
that students are acquiring more 
knowledge, they'll buy in. But if they 
feel that it's just one more fad, or one 
more thing they're being asked to do 
without support— a start-of-school- 
year seminar on teaching reading and 
nothing else, for example— they'll 
understandably resist. 

MSAN: What is the role of school districts 
in supportingthiswork? Whatcan thedis- 
trict do to support the type of classroom- 
level change you saw in schools that closed 
gaps in student achievement? 

Walsh Districts have a critical role to 
play in supporting the changes neces- 
sary to close achievement gaps. First, 
the district can make sure teachers 
have easy-to-use benchmarks and 
diagnostic assessments and then do a 
quick turnaround of the data these 
tools provide. 

Second, the district needs to fund the 
infrastructure to support teachers' 
understanding and use of data to 
inform their instruction. This includes 
funding professional development 
around best practices and, most 
importantly, peer-to-peer learning. 
Teachers need structured time to 
meet in subject- and grade-alike teams 
and talk about what the data mean 
and what strategies to use to address 
the needs of their students. Districts 
need to tap into the "gems" within 
their schools — highly effective teachers 
who should be freed up to coach and 
mentor other teachers. 



Finally, the district must focus on 
equity. Developing and coalescing 
around an equity vision provides clar- 
ity in terms of mission and priorities 
that ensure continued support for 
necessary work. Several of the schools 
in the study had supportive people 
within the district office but not a 
districtwide vision and commitment. 
While some schools in these districts 
were able to make gains, future school- 
level success is always in jeopardy with- 
out a deep, systemwide commitment. 

MSAN: In the cu rrent NCLB climate, what 
could policymakers learn from your study? 
What could they do differently- based on 
your research- to get the kind of results 
achieved by the schools you studied and 
that the law states as its intended goals? 

Walsh Policymakers should recognize 
both the value and the needs posed 
by NCLB. The value is that the 
achievement gaps are now on every- 
one's radar. Subgroups— such as 
English language learners— that have 
long been allowed to languish are 
now in the spotlight. What’s desper- 
ately needed, however, are increased 
supports to help teachers reach more 
students more effectively. Teachers 
need high-quality diagnostic data to 
inform instruction; they need access 
to best practices thro ugh professional 
development; and they need systems 
and structures to improve, such as 
classroom-based coaching, peer col- 
laboration time, and mentoring. Laws 
and policies are effective only if they 
change the technical core of teaching 
and learning. Both incentives and 
support are necessary for teacher 
to improve their daily classroom 
practice. ■ 

- Kiley Walsh is currently the Knowledge, Learning, 
and Results Manager at the Stupski Foundation in 
Mill Valley, California. She continues to research how 
schools and districts are closing the achievement 
gaps while raising student achievement for ail. 
Contact her at 415-384-2462 orkiieyw@stupski.org. 

A PDF version of After the Test is available on 
the Learning Point Associates website at 
www.ncrel.org/gap/studies/basrc.htm. A softcover 
print version can be purchased from Springboard 
Schools website at www.springboardschools.org/ 
prof_ dev/ resea rch_ studies.html. 
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Welcome Columbia Public Schools! 

MSAN would like to welcome Columbia (Missouri) Public 
Schools to our network of now 24 districts. We look 
forward to sharing, learning, and collaborating with our 
colleagues in Columbia as we work together to eliminate 
race as a predictor of achievement. 
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